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This paper explores the role of professional learning communities in facilitating effective 
use of data by school administrators and teachers. The paper draws upon three related 
research studies conducted in Milwaukee Public Schools. Grounded in systemic reform 
theory, a common goal of the studies was to build the capacity of school staff to use data 
more effectively for continuous improvement and decision-making to improve student 
learning. Knowledge management theory helps define what is meant by “effective use of 
data” in a systemic reform context, provides a useful framework for determining what it 
means to learn from data, and illustrates the successful use of data in school and 
classroom settings. This paper sheds light on what school staffs need to learn from data, 
the organizational barriers to learning from data, and how professional learning 
communities can contribute to improving the organizational culture and structure 
necessary for effective data use. Examples from recent research studies are used to 
illustrate why professional learning communities provide an ideal organizational structure 
to address both the challenges schools face and the needs of teachers as school staffs seek 
to learn from data and use it effectively to improve student learning. 
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Introduction 

This paper builds a case for endorsing professional learning communities as an 
important organizational structure to support systemic reform, organizational learning, 
and effective use of data by school administrators and teachers. It is organized in five 
sections that: 1) describe the systemic reform context of the research studies; 2) define 
effective data use and what it means to learn from data; 3) present research findings that 
identify the challenges and needs school staff face in learning from and using data; 4) 
provide examples of professional learning communities that successfully address the 
challenges and needs of school staff seeking to learn from and effectively use data to 
improve student learning; and, 5) demonstrate the important role of professional learning 
communities in facilitating systemic reform, organizational learning, and using data 
effectively. 

First, a brief overview of the development of systemic reform theory provides 
background for the research studies reviewed here. Research strands that emphasize 
district and school capacity-building for educational reform, and that recognize the 
importance of data in building human, social and structural capacity of schools, explain 
the focus of the research studies. The attributes of learning organizations and professional 
learning communities are then described. Similarities between the capacity-builders of 
systemic reform and the characteristics of learning organizations and communities of 
practice are highlighted. 

Second, knowledge management theory helps define what is meant by “effective 
use of data” in a systemic reform context, provides a useful framework for determining 
what it means to learn from data, and describes what the successful use of data in school 
and classroom settings looks like. 

Third, the findings from the research studies in Milwaukee Public Schools (MPS) 
are presented. Early research revealed that schools are not well organized to use data 
effectively. Seven challenges for schools to improve their organization to use data are 
identified. The results of teachers’ interviews are used to develop an understanding of 
how school staff members currently use and learn from data and what they regard as 
needing improvement. 

Fourth, the findings from our most current research with professional learning 
teams are summarized and examples are used to demonstrate how these teams address the 
key data challenges and needs identified by earlier research. 

Finally, the paper concludes by summarizing the role of learning communities in 
facilitating systemic reform, organizational learning, and effective data use. Professional 
learning communities are shown to address the expressed data challenges and needs of 
school staff, demonstrate how schools succeed in meeting systemic reform goals, and 
make effective data use and learning from data possible. 
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Theoretical Background and Research Base 

Systemic Reform Research Context 

Systemic reform theory is based on the belief that an aligned system of standards 
and instructional guidance at all levels of the educational enterprise is critical to 
improving the quality of teaching and learning. At the heart of systemic reform theory are 
standards that, when linked with other components of systemic reform — such as 
accountability, curriculum, instruction, professional development, assessments, school 
autonomy, school improvement, and support mechanisms from states and districts — serve 
to create more effective schools and higher levels of student achievement (Clune, 1998). 

Since its introduction in the early 1980s, systemic reform theory (and related 
research) has gained momentum and has expanded in breadth and depth (Fuhrman, 

1999). Studies of systemic reform policies and practice have evolved to encompass 
analysis of the components of reform at and between various organizational levels, the 
context in which the reform is taking place, and the capacity of educational organizations 
to change and achieve the goals of reform (Century, 2000; Clune, 1998; Furhman, 1999). 
Additionally, systemic analysis has moved beyond the state level to become an important 
lens for researchers seeking to understand how standards-based policy and practice play 
out at the district and school levels and influence teaching and learning in the classroom. 

Researchers have studied capacity as it relates to the ability of educational 
organizations to achieve systemic reform goals. Of particular relevance are studies of 
capacity that focus on an organization’s ability to learn by coordinating human, social, 
organizational, and structural capital and by making effective use of data. Spillane and 
Thompson (1997) show that local capacity for systemic reform can be thought of in terms 
of the organization’s ability to learn through building human, social, and financial capital. 
Features of human capital include knowledge, commitment, and disposition of local 
reformers. Social capital is defined as professional networks, trust, and collaboration, 
where those within the system learn from each other and from those outside the system. 
Financial capital refers to the local organization’s capacity to mobilize resources such as 
time, staffing, and materials for implementing change. Century (2000) essentially agrees, 
but employs four categories of capacity (compared to the three presented by Spillane and 
Thompson) to analyze systemic efforts: 1) intellectual proficiency and will (human 
capacity); 2) interaction, collaboration, and communication (organizational capacity); 3) 
fiscal resources (material capacity); and, 4) policies, procedures, and formal practices 
(structural capacity). 

Systemic reform researchers have continually been interested in assessing the role 
of data in developing and sustaining systemic improvements in student learning. 

Recently, however, a number of studies have shifted the analysis of the analytical 
capacity of systemic systems from a focus on using data for accountability to building the 
human and social capacity of schools to use data for school improvement, decision- 
making, and improving instruction. Diane Massed (1998) believes that data-based 
decision making is an important tool of educational improvement, but has found that 
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making data useful is one of the continuing challenges of building capacity in systemic 
reform. Massed is interested in understanding the influence of accountability mechanisms 
and student performance measures on teachers and administrators. Key to developing this 
understanding is investigating how teachers and administrators interpret and use 
performance data and how that translates into action. Massed suggests that most 
accountability models assume that practitioners understand what performance data mean 
and that they have the knowledge and skids to translate these data into appropriate action. 
Massed’s research on the National Science Foundation’s Statewide Systemic Initiatives 
(1998) noted weaknesses in both of these assumptions. She reports that teachers and 
administrators are not trained (a) to interpret test statistics or the relevance of data to 
school improvement or (b) to use data to identify strategies that will improve instruction. 
Fuhrman Q999) also supports the use of data for monitoring organizational processes — 
reform as implemented — suggesting that reforms must be linked to outcomes designed to 
achieve continuous improvement of both elements. 

Improving the organizational capacity of districts and schools to coordinate 
resources around coherent goals of student achievement and to use data as a guide for 
instructional improvement is the focus of the research we conducted in the Milwaukee 
Public Schools. Grounded in systemic reform theory, a common feature of each study 
was to build the capacity of school staff to use data more effectively for continuous 
improvement and decision making in the effort to improve student learning. 

Learning Organizations and Professional Learning Communities 

External influences such as increases in required testing, emerging accountability 
systems at state and district levels, and the 2001 revisions to the Elementary and 
Secondary Education Act (ESEA)^ have put pressure on school staffs to understand and 
respond to a growing set of data on school and student performance. Similarly, reform 
efforts typically require schools to evaluate the implementation and effects of new 
strategies or programs. Internally, school administrators are looking for new ways to 
increase the capacity of their staffs to reflect upon and use data to improve instruction 
and learning. Schools are being encouraged to become “learning organizations’’ and to 
develop transformational approaches to school improvement through collaboration, 
inquiry, and continuous improvement. One approach is to build professional learning 
communities at the school to facilitate organizational change and educational 
improvement among staff members and across the school. A brief review of the literature 
reveals the common attributes of learning organizations and professional learning 
communities and highlights the important role data can play when used in these 
organizational structures to build capacity for school-level educational improvement. 

Current literature recognizes that an important key to developing capacity for 
educational improvement lies in the successful development of the school as a “learning 
organization” (Fullan, 1993). For a school to be a learning organization, schools must 
overcome fragmentation in their reform efforts, form alliances and partnerships outside 

' No Child Left Behind Act of 2001 (Public Law 107-1 10, 1 15 Stat. 1425; reauthorization of the 
Elementary and Secondary Education Act; httD://www.ed.gov/offices/OESE/esea/) . 
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the school, solve problems collectively, focus on changing teaching and learning, and 
develop shared values and beliefs about learning and change, all while remaining 
receptive to learning on a continuous basis within a changing educational environment 
(Fullan, 1993). Educational policymakers, researchers, and practitioners agree that this 
kind of school transformation requires more than just the restructuring of schools (i.e., 
changing organizational hierarchies, governance structures and planning groups); it is 
also about “re-culturing” schools by creating new ways of thinking and doing among all 
members of the staff (Louis, Marks, & Kruse, 1996; see also Scribner et al., 1999; Baker, 
DuFour, & Burnette, 2002). Louis, Marks, and Kruse write that such learning 
organizations are characterized by “shared norms and values, collective focus on student 
learning, collaboration, deprivatized practice and reflective dialogue.” Newmann, King, 
and Youngs (2000) assert that improvements in student achievement are most affected by 
instructional quality and that improvements in instructional quality require both 
promoting teacher learning and developing school capacity: 

Researchers tend to agree that to promote the kind of teacher learning that 
leads to improvement in teaching, professional development should 
concentrate on instruction and student outcomes in teachers’ specific 
schools; provide opportunities for collegial inquiry, help and feedback; 
and connect teachers to external expertise while also respecting teachers’ 
discretion and creativity. Finally, these experiences should be sustained 
and continuous, rather than short-term and episodic (p. 259). 

Newmann et al. (2000) also argue that adherence to these points enhances 
individual professional development, but conclude from their research that effective 
schools must also develop these points in conjunction with school organizational 
capacity. Among the aspects of school capacity they regard as important are the 
development of professional community and teachers’ knowledge skills and abilities. 
King and Newman (2000) reiterate these findings in another article, writing that, to 
develop their “knowledge, skills and dispositions,” teachers need to be engaged in “an 
organized, collective enterprise” such as professional communities. 

In their efforts to become learning organizations, schools often set up professional 
learning communities. Professional communities are school-based, teacher-centered 
organizational structures that are linked to organizational culture in such a way as to 
promote organizational learning and improvement in schools (Scribner et al., 1999). 
Structurally, professional communities vary depending on the size and complexity of the 
staff, scheduled planning time, and level of teacher empowerment at the school. How the 
school organizes and increases its human and social resources — such as innovation, 
leadership, respect, feedback, and staff development — also influences the makeup of 
professional community. Professional communities affect organizational culture by 
creating an environment where teachers are grounded by their shared values, beliefs, and 
dispositions, are continuously learning, and are critically reflective (Louis, Marks, & 
Kruse, 1996). A short review of the literature reveals a set of common attributes of 
professional communities. Professional learning communities are: 
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• Inquiry-based 

• Focused on student learning 

• Goal- and results-oriented 

• Collaborative 

• Reflective 

• Based on shared values and beliefs 

• Committed to continuous improvement 

(Fullan, 1993; Murphy & Lick, 2001; Faker, Dufour, & Burnette, 2002; King 

& Newmann 2000; Glickman, 2002; Brandt, 2003). 

Integral to the successful functioning of learning organizations and professional 
communities is the use of data and information to enhance inquiry, continuous 
improvement, reflection, and learning. In their guide to building professional community. 
Faker, Dufour, and Burnette (2002) encourage professional learning communities to 
become “research-based” and “data-driven.” Similarly, Brandt (2003) encourages 
learning organizations to “gather, process and act upon information,” as well as to 
exchange and get feedback on that information. Murphy and Lick (2001) stress the 
importance of making data-based decisions throughout the inquiry process to inform 
planning, implementation, and evaluation. Schmoker (2001) provides specific examples 
of how schools and districts have integrated the use of data and information into their 
learning communities, specifically noting the importance of “teachers reviewing 
assessment data for the purpose of improving practice.” Feldman and Tung (2001) report 
that teachers change their attitudes and practices in schools that use data-based inquiry; as 
a result, a more professional culture emerges, facilitating professional dialogue and 
reflective practice. Clearly, the use of data contributes significantly to the process of 
learning and improvement in professional communities and learning organizations. 

The literature on learning organizations and professional communities 
demonstrates that a large part of a school’s capacity to be effective and improve depends 
on how it is organized to learn. A school’s capacity to learn is defined by a combination 
of its human, social, structural, and organizational capital (see Century, 2000). The 
common attributes of school learning organizations and professional learning 
communities provide the structure and culture conducive to organizational learning by 
focusing on the following: teaching and learning; collaboration among staff and with 
external partners; inquiry-based learning and reflection, shared values, norms, and 
dispositions of teachers, and a commitment to continuous improvement. When imbued 
with data and information, learning organizations and professional learning communities 
can maximize these attributes to positively affect professional learning, educational 
change, and reform. 

Learning from Data 

What do “learning from data” and “effective data use” mean? Knowledge 
management theory helps define these terms in a systemic reform context and provides a 
useful framework for determining the successful use of data in school and classroom 
settings. 
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Knowledge management theory is based primarily on the research of business 
schools in the United States and Europe seeking to develop better ways for organizations 
to manage information, develop and sustain complex information technology systems, 
and learn from data. Laura Empson (1999) explains the continuum of data, information, 
and knowledge. She contends that knowledge is a product built from data and 
information: 

Data are objective facts, presented without any judgment or context. Data 
becomes information when it is categorised, analysed, summarized, and 
placed in context. Information therefore is data endowed with relevance 
and purpose. Information develops into knowledge when it is used to 
make comparisons, assess consequences, establish connections, and 
engage in a dialogue. Knowledge can, therefore, be seen as information 
that comes laden with experience, judgment, intuition, and values, (p. 8) 

Thom (2000) suggests that the transformation of data to knowledge is an 
important knowledge management strategy for decision makers at all levels of the 
educational system — including school-level administrators and staff. Knowledge 
management strategies add value to data, and when data is transformed into knowledge, 
educational systems are better able to manage complex information, use data for 
decision-making, and improve systemic reform efforts (Thom, 2002). Knowledge 
management can also be thought of as the effective use, or application, of data. Thom 
points out the importance of organizational culture in enabling or blocking the use of 
knowledge. Citing Davenport and Davenport (1999), he highlights their suggestion that 
the willingness to engage in problem-solving processes and to share information with 
“outsiders” is an important resource for enabling knowledge management efforts. The 
Davenports conclude that cultures that support knowledge accumulation and application 
will produce the most effective, efficient organizations. 

Fullan (2001) also references the knowledge management literature, noting the 
importance of the concepts of knowledge building, knowledge creation, and knowledge 
sharing for educational organizations. Fullan transfers information technology and 
business concepts into educational settings. He suggests that information is best 
transformed into knowledge and learning in a social context. It is important that the 
exchange and sharing of information take place in the context of the school: “ ... it is 
local networks that count, because it is when we are learning in context that knowledge 
becomes specific and usable” (p. 104). Fullan concludes that district and school 
organizations with collaborative cultures are poised to tap into information and the tacit 
knowledge of professional staff, which can be enhanced through exchange with similar 
organizations. 

Taken together, systemic reform theory and knowledge management theory help 
define what it means for school staff to “learn from data” and “use data effectively.” 
Learning from data means to transform data into information as it is interpreted in 
context. Data then becomes knowledge as it is shared, applied, and used to promote 
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change and improvement throughout an organization. Knowledge creation becomes 
systemic when it is shared within the school and across the system to raise the 
effectiveness of reform efforts and improve teaching and learning. Fundamental to each 
of the research studies we conducted was the belief that the capacity of schools to 
improve local decision making, instructional guidance, and student achievement and to 
sustain school improvement would be enhanced through the systemic use of data within 
such a learning environment. 

Findings from Milwaukee Public Schools 

Research Purpose and Methods 

The analysis draws upon the three Milwaukee research studies, each supported by 
grants from the Joyce Foundation, and conducted in Milwaukee Public Schools (MPS) 
over the last six years: The Center for the Study of Systemic Reform in Milwaukee Public 
Schools (SSR-MPS), The Study of Electronic Information Systems in Milwaukee Public 
Schools, and Strategies and Models for Improving Student Learning through Effective 
Data Use (SMILE). MPS includes nearly 100,000 students and 12,000 staff members at 
more than 200 schools. Over 80% of students are non-White; over 70% of the students 
are eligible for free or reduced-cost lunch; the mobility rate is nearly 30%; and, the high 
school attendance rate is only 77%. 

A common purpose of each study was to increase the capacity of district and 
school leadership and of teaching personnel to participate more meaningfully in decision 
making by improving their analytical capacity and training them to use data to link 
planning and evaluation to instructional practices designed to improve student learning. 
The first study incorporated research, at both district and school level, on uses of data for 
school improvement as a component of systemic reform. The next two studies were 
specifically planned to simultaneously study and increase the capacity of school-level 
staff members to effectively use data for improving teaching and learning. The goal of 
our third and on-going study (SMILE) is to produce models that demonstrate how school 
staff can develop the organizational, technical, and analytical capacity to measure school 
improvement and instructional practices and to apply what they learn in a continuous 
improvement process to both teaching and learning. 

All three studies used a design-based research model we have termed “embedded 
research,” in which an interactive partnership between practitioners and the research team 
blends research and technical assistance to develop and study innovations and 
interventions in context. This methodology yields greater involvement of both parties and 
generates a deeper and more precise understanding of the issues and how they are 
connected in a framework of systemic reform. This leads to the identification of strategies 
and the formulation of models that are the most plausible pathways to improvement. 
While similar to design experiments conducted by Ann Brown (1992), and to other 



^ For more information on design-based research, see: Kelly, A. E. (Ed.). (2003). The role of design in 
educational research [Special issue]. Educational Researcher, 32(1). 



er|c 



8 



9 



Learning from Data: The Role of Professional Learning Communities 



Sarah A. Mason 



design-based research methodologies that study forms of student learning, our approach 
focuses the design-based methodology on adult learners in order to study the elements of 
school capacity for improvement and educational reform. 

A total of 1 1 schools (5 elementary schools, 4 middle schools and 2 high schools), 
over 60 teachers, 12 principals, and 12 school learning teams have participated in the 
three research studies over a six-year period (1998-2003). Demographics, school 
characteristics, accountability reports, and capacity information were collected for each 
of the 1 1 schools selected for participation in the studies. At each school, a series of 
baseline interviews and focus group sessions were conducted to acquire qualitative 
information on the school’s capacity for data use. A “capacity instrument” for measuring 
school organizational, technical, and analytical capacity to use data for decision making 
was developed for use in one of the studies. On site visits, we provided technical 
assistance and gathered capacity information at regular intervals, charted any changes, 
and documented progress and problems. We used the capacity instrument, field 
observations, and documented interactions with the school team to obtain a detailed 
understanding of how data were being used at the school. 

Qualitative research methods were used to gather and analyze data from multiple 
sources, including interviews, focus groups, observations, and the collection of 
documents and artifacts. Semi-structured interviews of school staff participants were 
audio-taped and transcribed. Each technical assistance site visit, all school observations, 
and telephone communications with school staff were documented in field notes. Focus 
group meetings were documented in a self-assessment data capacity survey document 
and field notes. Two researchers were present at each site visit, and each completed a set 
of post-visit field note summaries using a semi-structured protocol. Interview data and 
field notes were analyzed using a combination of Microsoft Access and Nudist software. 
We collected copies of each school’s school improvement plan, state and district 
assessment results, and demographic information, as well as relevant school and 
classroom data such as classroom assessments, curriculum guides, writing prompts, and 
rubrics. 



Our research methodology was designed to identify capacity issues, 
organizational problems, and the needs of district and school staff in using data. Initially, 
the research focused on developing an understanding of the district’s information system 
and of its analytical capacity to use data for school improvement. In later studies, we 
worked with individual schools to learn more about their challenges and needs when 
using data. At the school sites, we field-tested the use of learning teams in addressing 
these problems and needs. 

What do schools need to use data effectively? 

Data obtained from the first two research studies was used to identify the 
systemic, organizational, and technical challenges schools and their staffs must address as 
they build their capacity for using data. 
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In a systemic review of analytical capacity, all levels of the system must work in 
concert to meet the goals of reform. For that reason, we began our research for the first 
study of school data use at the district level, looking at the district information system and 
decision-support structures to learn how they were designed to assist schools in using 
data for decision making. We identified several systemic challenges facing MPS and 
those schools that wanted to use data for reform and school improvement. While noting 
that collaboration is necessary for innovation in knowledge management and systemic 
reform, Thom (2002) found that differences in data needs and uses across the various 
levels of the district system presented barriers. In 1996, the district needed an information 
system to manage a complex set of administrative, demographic, and assessment data on 
over 100,000 students acquired from more than 200 schools. In the Technology Strategic 
Plan (Milwaukee Public Schools, 1996), the district outlined specific data needs for 
teachers and school administrators who needed a data system for school and classroom 
management, but who also wanted to use the data system to study and evaluate student 
performance and school improvement. The district’s information system was originally 
conceived with these school-level data uses in mind, but when designing and 
implementing the data warehouse and transactional systems, barriers were encountered 
that led to a compliance-oriented information system that held little value for schools. 
Schools had difficulty accessing data in the system, and the data that were available were 
not in a convenient format for school and classroom improvement use. Moreover, many 
of the data were displayed in an aggregate format that did not allow for extensive analysis 
by district staff. The limited access of school staff to the data allowed for even less 
meaningful analysis (Clune & Webb, 1999). 

Recently, the systemic links between the district, the district information system, 
and the schools become more aligned around schools’ needs and uses of data. Recent 
improvements have made the information system more accessible to district 
administrators and school staff members alike. More data is kept in individual student 
records and many staff members have been trained in using BRIO software to retrieve 
data specific to their school and students. The MPS Division of Assessment and 
Accountability has increased the amount of data assistance it provides to schools by 
conducting professional development data workshops, electronically transferring data to 
schools, providing technical assistance for analyzing and applying data, and conducting 
more in-depth research and analysis of school and student performance data. The data 
focus has moved from district reporting and compliance to include school improvement 
and student achievement. Training and technical assistance have become more 
collaborative and school improvement planning now emphasizes the use of data for 
continuous improvement. Accountability reports have expanded to include results of 
annual testing, as well as both school and student performance and progress. Professional 
development in assessment literacy has reinforced school-level understanding of state 
standards and has increased understanding of the primary data sources used to determine 
student promotion and school improvement progress. The district has trained school 
leaders in developing professional learning communities to incorporate the analysis and 
application of school data in planning and evaluating change efforts at the school level. 
The MPS information system is now more systemically organized and, with the proper 
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supports, promises to provide some of the necessary ingredients for effective data use 
(Clune & Webb, 2001, 2002; Webb, 2002). 

In our second study of school-level data use, we found that it was necessary for 
both school administrators and teaching staff to develop the processes and skills to 
become more savvy consumers of data. We found that individuals varied greatly in their 
comfort and knowledge when acquiring, analyzing, and using data. We were most 
successful in encouraging schools to use data when we were able to establish a formal 
problem-solving, program-evaluation, or action research orientation that required school 
teams to select a focus, frame questions, collect and analyze relevant metrics and data, 
and translate reports into action plans with a continuous improvement focus. 

In working with the six Milwaukee schools in this study, we learned that to be 
effective data use must become an active part of school planning and improvement 
processes, and it must become infused and accepted in the school culture and 
organization. Additionally, school staff members must develop the analytical capacity to 
understand and strategically apply data. Once fully integrated into a school’s systems, 
data can be transformed from mere numbers to useful information, which can then 
contribute to the staffs knowledge in effective and meaningful ways. 

The application of data to decision-making presents an array of complex 
challenges for schools. These challenges must both be addressed initially and attended to 
continuously if a school is to make successful and effective use of its data. We have 
identified six challenges schools need to confront as they build their capacity for using 
data for decision-making: I) cultivating the desire to transform data into knowledge; 2) 
focusing on a process for planned data use; 3) committing to the acquisition and creation 
of data; 4) organizing data management; 5) developing analytical capacity; and, 6) 
strategically applying information and results (Mason, 2002). 

What do teachers need to learn from data? 

At the start of each school-level research study, we interviewed school 
administrators and teaching staff members who planned to participate in the study. More 
than 50 teachers were asked to respond to questions regarding their current data use, and 
to identify their data-related skill development and resource needs. They spoke of the 
current pressures on schools and teachers to understand and use data and suggested better 
ways to motivate school staff. Analysis of these teacher interviews across all three studies 
has revealed additional organizational needs and challenges specific to teachers’ capacity 
to learn from data. 

When asked about their current use of data, teachers responded that they use data 
in a variety of ways: 

• To improve and evaluate instructional strategies 

• To write grant applications 
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• To identify student strengths and weaknesses, areas of need, progress, and 
proficiency levels 

• To organize “anecdotal evidence” 

• To communicate with and motivate other teachers 

At the same time, teachers responded that their use of data was infrequent, often random, 
and conducted in isolation from other teaching staff. They often characterized their data 
use as one-time occurrences, sometimes associated with a specific purpose in mind, such 
as obtaining grant money, or writing a master’s paper. While they reported that they 
sometimes shared the information with other teachers, it was never referred to as a 
common or regular practice. Often teachers referred to the one teacher or administrator in 
the school who was a “data expert,” noting that the rest of the staff relied on this person 
and did not use data themselves. Moreover, the data used by teachers was often limited to 
paper reports of state and district test results. Few teachers collected data in their own 
classrooms, sought out demographic or behavior data on their own students, or compared 
classroom assessment results to school, district, or state assessment results. Typically, use 
of data was not comprehensive throughout the school, nor was it consistent or 
continuous. 

When asked what additional data they would like to have, or what data they 
would find useful, teachers responded that they needed: 

• School- and classroom-level data 

• Data with an instructional focus 

• Disaggregated data 

• Individual student data 

• Longitudinal data and historical records 

• Behavior and attendance data 

• Student home and background data 

• Quick access to timely data 

Teachers wanted to use these data to identify students’ needs both inside and outside the 
classroom. They wanted to use the information to target their instruction and assistance to 
students. As one teacher responded: “The problem I see is kids not learning, and if data 
could zero in on where we can do the most good, in time . . . ”^ Others responded that a 
wide range of data was already collected at their school, but that they needed help in 
accessing and using it: “We need data that we can manipulate in the school and that can 
be accessed.” 

These types of responses led the way to a discussion of the skills and resources 
teachers felt they needed to use data more effectively. Teachers recognized that their skill 
needs covered a wide range, with most falling under the following categories: 



^To protect confidentiality, the names of teachers are not given. All quotes from teachers are taken from 
interviews conducted during the course of our research over the last six years. 
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• Assessment literacy 

• Alignment among assessment, curriculum, instruction, and standards 

• Technology (computers, software, databases) 

• Data skills (management, analysis, and application) 

• Identifying strategies for change 

Teachers spoke of wanting to learn these skills in order to: improve learning in the 
classroom and school; identify strengths and weaknesses in curriculum and instruction; 
target student needs; ask better questions and get better answers; measure student 
progress towards proficiency; create visual representations; communicate with students, 
parents, and other staff; and identify strategies for change. 

In order to learn these skills and improve their ability to use data effectively, 
teachers recommended that the following resources and supports be provided: 

• Professional development 

• Time to work with data both individually and collectively 

• Access to data 

• Administrative support 

• External sources for strategies and research-based practices 

Having time to learn about and use data was critical to many teachers. Without support 
from school administrators, including a commitment to provide time for teachers to learn 
how to use data, they felt their personal use of data would remain sporadic. They wanted 
time, in school, to learn with each other how to use data and share results. Using data 
alone, in their classroom, was not considered to be efficient or an effective use of their 
time. They noted that the process for using data had to be user-friendly and provide 
meaningful and useful feedback to teachers. They spoke also of the need for professional 
development for teachers to learn about data use and the need for greater opportunities to 
incorporate data into their current planning and development. 

Teachers also talked about their motivations for and against using data. External 
incentives driving data use were referred to largely as “pressures.” Teachers referred to 
pressures to improve student test scores to meet new accountability guidelines under 
national, state, and district accountability systems. Many talked about the district’s 
student proficiency, promotion, and graduation systems as placing new responsibility on 
teachers to improve student performance. They reported that being able to interpret the 
assessments and assessment results to understand where they and their students “stood” 
has become increasingly important. They spoke of needing to know the strengths and 
weaknesses of student performance so they could identify students’ learning needs in 
time to bring them up to proficiency. These external pressures to use data were mostly 
considered to be negative and were referred to as a “burden” rather than as positive 
motivation to learn. However, most teachers recognized the importance of using data and 
the role incentives might play in motivating teachers’ data use to improve teaching and 
learning at their schools. One teacher captured the general feeling expressed by many of 
those interviewed, and suggested a solution: “You’ve got to have motivation, and I think 
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it comes from the training of the teachers.” Most of the solutions suggested to motivate 
teachers were incentive mechanisms — requiring internal responses by the school: 

• Professional development to increase both “skill and will” 

• A shift in emphasis from individual improvement to organizational 
improvement (i.e., a shared process for continuous improvement) 

• Instructional and school improvement focus 

• Alignment with standards and curriculum 

• Leadership support for time and resources 

• Recognition for progress 

In these interviews, teachers explained that in order to learn from data and use it 
effectively, four general areas of need must be met: 1) Data inquiry needs to be a 
collaborative, shared process; 2) data need to have a leaming/instructional focus; 3) 
professional development needs to focus on assessment literacy, alignment, decision- 
making processes, data skills, and strategy identification; and, 4) school-level 
organizational mechanisms must be in place to provide incentives and support for data 
inquiry, instructional improvement, professional development, and continuous 
improvement. 

The Role of Professional Learning Communities in Using Data: Examples from 
MPS 



In preparing for our most recent study, we reviewed research findings from our 
previous MPS studies in order to address the expressed needs and identified challenges in 
building analytical capacity for systemic reform. From our first study, we identified the 
systemic challenges the district and schools faced in building analytical capacity and 
coordinating effective data use. In our second study, we conducted case studies of six 
schools to identify their data needs and the organizational and technical barriers they 
faced in using data effectively. The findings from both of these studies enabled us to 
frame our research and design for the third study. The focus of this ongoing study is to 
identify the characteristics and conditions that lead to effective data-use in schools. 

In this section, we use the findings from all of our research as a basis for 
investigating the role of professional learning communities in supporting systemic 
reform, organizational learning, and effective use of data by school administrators and 
teachers. Our research identified systemic challenges, school barriers, and teacher needs 
facing schools committed to learning from and using data effectively. Systemic reform, 
organizational learning, and knowledge management theories provide an analytic 
framework for understanding the components and attributes necessary for meeting these 
needs and challenges. From our current research, we draw three examples of emerging 
professional learning communities in MPS schools that address the systemic issues, 
school challenges, and teachers’ needs in using data effectively. The examples shed light 
on how learning communities function in promoting effective data use. 
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Building Systemic Data Capacity: Human, Social, Structural and Organizational Capital 

The South Middle School Example‘S 

South Middle School is a good example of how a school that has a history of 
using data primarily for accountability can increase its systemic capacity to use data by 
implementing a professional learning community. South has a history of using data in 
four distinct and separate ways: 1) School administrators collected paper reports on 
school demographics and district and state test results and used them once a year to 
respond to district calls for accountability reporting and as a basis for writing grant 
proposals; 2) student attendance, discipline, course assignments, and grades were 
collected by the office staff and entered into the student transactional data system; basic 
frequency reports and individual student records were retrieved; 3) an in-school data 
expert set up a database to track student scores on the district proficiency and 
performance assessments — these data were used primarily to report back to the district on 
8'*’ grade students’ promotion; and, 4) a few individual teachers reported surveying their 
students to obtain feedback on in-class instruction and using an assortment of classroom 
assessments to identify individual student learning needs. Staff reported that data was 
used infrequently and in isolated ways for specific purposes; there was little sharing of 
data among staff or between administration and teachers. The focus of their data was on 
student and school accountability — it was used to track student behavior and determine 
promotion and to document school performance and compliance reporting. At the start of 
the 2002-2003 school year. South was identified as one of 56 Milwaukee “schools in 
need of improvement” as defined by the state of Wisconsin and the revised Title I of the 
No Child Left Behind legislation.^ This designation put added pressure on the school to 
monitor their student performance data and accountability measures. 

With this data history and the current mindset instilled by the need to improve 
student test scores, the staff at South began participation in our study. One of their first 
tasks was to form a team of participants. South formed a rather large and inclusive team 
consisting of a cross-section of the school’s staff, including the principal, assistant 
principal, technology coordinator, literacy coach. Special Education coordinator, and a 
number of teachers from different subject areas. At the start, the team’s main goal for 
using data was to learn how to set up a database to disaggregate state and district test 
scores and identify grade levels, student groups, and subject areas needing attention. 
Under their current accountability pressures, they were initially concerned only with 
improving their students’ test scores on the state assessments. However, over the course 
of the study, the team shifted its focus from using data solely for accountability purposes 
to using data at a deeper level. The team wanted to learn more about the assessments — 
what objectives were measured in each subject area and how those aligned with school 



The names of the middle schools and elementary school are fictitious. 

^ Milwaukee Public Schools, Office of Research & Accountability. (2002) Milwaukee Public Schools' 
2001-2002 Report Card: 2001-2002 District Report (December 3, 2002). Milwaukee, WI: Author. 
Available at: http://www.milwaukee.kl2.wi.us/fileBroker.php/4942/3-District.pdf [Accessed April 3, 
2003]. 
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curriculum and state standards. Team members decided this information would help them 
interpret their test scores in a more meaningful way and would allow them to better 
identify the strengths, weaknesses, and gaps in their students’ learning and in their 
curriculum. They also chose to look more closely at their classroom-based assessments 
(used to decide 8'** grade promotion) to see how these aligned with the state standards, the 
curriculum, and student performance on state tests. They used this information to monitor 
change and progress in student performance, identify areas in their curriculum that 
needed supplementation or improvement, and to evaluate the effects of new strategies on 
performance. 

These goals for learning from data mark a shift in the team’s thinking about data. 
They were no longer seeing data solely as a source for accountability and compliance, but 
as a tool for building an information and knowledge base about teaching and learning at 
the school. This transformation in thinking came from a variety of sources: the 
collaborative setting of the team (which allowed new voices to be heard and new ideas to 
emerge), our requests to select a “focus” for their data inquiry (they chose to look at 
mathematics achievement), and their need to understand data at a deeper level in order to 
inform reflection, action, and improvement. The team’s approach to data use began to 
develop and take on all the key characteristics of a professional learning community. It 
was a new way of thinking that reflected the beginning of a change in the school’s 
organizational culture as it viewed, applied, and valued data. Moreover, the establishment 
of a professional learning community began to change the organizational structure of the 
school — particularly in regards to how the school was organized to learn from data. Data 
were no longer managed by one data expert, or kept in separate departments for discrete 
uses. Instead, data were being developed and retrieved for use by many teachers in the 
school and for a wide range of purposes. 

The professional learning community has helped South to build its capacity to use 
data. Spillane and Thompson’s (1997) and Century’s (2000) concepts of capacity provide 
a useful way to assess South’s progress in building its capacity to use data. South has 
increased its human capital by shifting its commitment to use data from accountability to 
instructional improvement. Teachers’ and administrators’ dispositions toward data have 
changed for the better, as they now see value in using data to meet their own needs, rather 
than as a descriptive tool or as a source for meeting compliance. We saw an increased 
interest in and readiness among staff members to use data and to see it as a legitimate 
source for learning. The social capital of the school changed as staff began to seek 
additional data and technical assistance from the district and from us as researchers, and 
began to relate to each other in a sort of professional network. The organizational capital 
for using data also improved, as the team members increased interaction and 
collaboration with each other by interpreting data, discussing its implications, and sharing 
feedback with others. Finally, South has begun to build structural capacity around the use 
of data. At mid-year, the “data expert” at South transferred out of the school, which 
meant the school lost much of its vested capacity for using data. However, through their 
work on the study, team members have recognized the importance of distributive 
leadership, resources, and formal mechanisms for using data. They have begun to 
reallocate their data expertise and resources and are rethinking their school-wide policies 




16 



Learning from Data: The Role of Professional Learning Communities 



Sarah A. Mason 



and practices around assessment data. The team has started to align its curriculum and 
assessments to the state standards, interpreting objective performance and item analysis 
results from the state tests, and now discusses and shares results with teachers as they 
plan strategies to address weaknesses in student performance and curriculum. South has 
improved its use of data to inform and align the components of systemic reform. 

While the installation of a professional learning community was integral to the 
process of building data capacity and systemic reform capacity at the school, much work 
still needs to be accomplished. Even though there is an improved sense of purpose and 
motivation toward using data more broadly in the school, tensions between the 
administration and staff still remain and the school does not yet have shared values and 
direction regarding reform. It is not clear whether the professional learning community is 
ingrained enough in the school’s culture or organizational structure to help build 
sufficient human capital and consensus for reform. Social capital at the school has 
increased with renewed interest in aligning the school’s classroom-based assessments 
with the district’s new learning targets. Yet, this interest mainly resides with 
administrators; the teaching staff has not “bought into” the idea of the need for alignment 
and consistent measures of student performance. As yet, many staff members have not 
consistently attended team meetings, and it is uncertain whether the network that was 
beginning to emerge during the study will become institutionalized in the building. Staff 
may resist efforts on the part of administration to formalize the classroom assessment 
process. The organizational and structural capital the school has built over the last year is 
fragile as well. While the professional learning community promoted interaction, 
collaboration, and sharing around the use of data, it is far from certain that the team will 
be able to expand this interaction beyond the scope of the project. Finally, maintaining 
structural capacity will depend on school leadership to support, maintain, and allocate the 
time and resources for distributed ownership of data. 

Addressing School Challenges 

The North Middle School Example 

North Middle School could be described as a “data savvy” school in that it has 
utilized data in a variety of unique ways for a number of years. For example, three years 
ago. North began to analyze “event-based” data, a term used to describe data that refers 
to a specific incident or action, rather than to a test score or student demographic 
variable. By tracking the patterns of events such as discipline referrals and attendance 
infractions, school staff learned more about where, when, and how often certain events 
occurred. The principal provided summary information on a variety of events that 
occurred during the semester to teams of teachers. These summaries were used to 
encourage discussion and create understanding about the patterns of student behavior and 
teacher behavior management practices at the school. The data were also used to support 
decisions about resource allocation — e.g., the school planned to hire an additional 
counselor to help students who encounter out-of-school difficulties that staff members 
believe affect student behavior at school. 
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North Middle School is organized by family teams, to which the principal 
presents his findings and monthly reports. The school relies primarily on the principal for 
collecting, managing, and presenting these data. The school is working on organizing 
MPS Educational Plan data so that growth and change over time can be analyzed, but 
currently has no system for analyzing gains in classroom data. In the 2002-2003 MPS 
Educational Plan, North identified a need to improve reading instruction school-wide and 
developed target goals to measure the progress of change and the impact of their 
improvements. The analysis methods, limited by the data available, primarily relied on 
comparing school averages to the district and state averages, and on disaggregated 
averages and frequencies. Typically, their MPS Educational Plan and classroom learning 
data were contained within subject areas — there was not much “cross-talk” regarding the 
results between teams or among teachers. Staff members reported that they were not as 
successful at analyzing and using performance data as they were in using process and 
behavior data. While much process data were collected, maintained, and reported to 
school staff on a monthly basis, staff members say that these reports spurred limited 
discussion and were infrequently applied to specific questions or problems. While data 
were used to guide change in school and classroom processes two or three times a year, 
school-wide change strategies and interventions were evaluated only once a year. 

As part of the goal in participating in our study, the North principal expressed a 
desire to expand the school’s use of data to the family and classroom level. His desire 
was to increase teacher ownership of data and to improve the data culture of the school. 
Uppermost in his mind was the need to make the data accessible, user-friendly, and 
meaningful to teachers. It was important, too, to move beyond the “event-based” process 
and behavior data to use student-learning data to contribute to teacher learning about how 
to improve classroom instruction. To accomplish these goals, the principal asked for 
interested teachers and staff members to join the study. The resulting team members 
decided to focus on learning more about their students’ reading performance and 
progress, a major goal in their 2002-2003 MPS Educational Plan. The team split into two 
work groups. One organized a database of all the existing reading-related data to analyze 
current reading performance status of students; the database will also be used for future 
data collection and analysis. The other work group designed and conducted a survey of 
teachers’ use of reading strategies across the curriculum. The purpose of the survey was 
to evaluate current practice and identify a set of effective strategies to use consistently 
across families and subject areas. They plan to implement these strategies next school 
year (2003-2004) and to both evaluate their progress (by surveying the teachers again) 
and evaluate the strategies’ impact on student reading (by analyzing pre-post reading test 
scores). The goal was to improve reading performance in every subject area by using 
strategies endorsed by teachers and by promoting consistent strategies throughout the 
school. While the work groups performed their tasks separately, they met regularly to 
share information, offer suggestions, and provide feedback. 

Through our case study research, we have identified six challenges schools need 
to confront as they build their capacity to use data for decision-making. In our study. 
North was able to address each of these challenges with a degree of success. Below is a 
summary of each challenge and an explanation of North’s approach to meeting the 
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challenge. We also highlight the role of its professional learning team in helping the 
school meet the challenges of using data. 

The first challenge is for schools to create a school culture that not only 
encourages the use of data, but that looks upon data as a source of information that can 
contribute to problem-solving and knowledge-building. It is essential for school 
leadership to gather support, commitment, resources, and direction to ensure that school 
data efforts are a success. Building this type of data culture or environment within a 
school may require significant professional development and frequent meetings with staff 
in which data play a role. North Middle School entered our study with this purpose in 
mind — to improve the data culture in the school by getting a greater number of teachers 
involved in using data and by developing new ways to put meaningful data into the hands 
of all teachers in the school. The project team — a professional learning community in 
itself — was supported by the principal with resources, time, and technical assistance to 
learn how to access and use the current databases. The goal of the work groups was to 
collect and analyze data and to share the results with the entire school staff to improve 
instruction and track the progress of students in reading. By sharing and using data with 
the entire school, all staff members could become more informed data consumers and 
learn to turn it into useful knowledge for problem solving and improvement. 

Second, it is important for schools to focus their use of data by linking it to their 
school planning and decision-making processes. A focused approach saves time and 
effort, allowing for more efficient use of limited data. An approach that aligns data 
inquiry to school planning and decision-making processes from the start is more likely to 
produce answers to specific questions, provide evidence to support school goals, and 
utilize information that can shed light on identified problems. Planned and targeted data 
inquiry can help to keep data analysis on track, as well as ensure that information is fed 
back into the planning process and that key decision-makers get the answers they need. 
The North team benefited greatly from identifying a specific focus for its work. Members 
selected a focus that crosscut the entire school (reading is not a specific course in the 
middle school, but rather taught across the curriculum). By starting with the school’s 
MPS Educational Plan and identifying the area of greatest need, they ensured that the 
work would have a results orientation and be aligned with their school goals. The focus 
on reading also helped each of the work groups identify relevant questions to frame their 
data collection and analysis. 

Third, acquiring the right data means becoming more familiar with the district’s 
and the school’s own information systems and databases. To get the specific data needed, 
schools have to create new data sources and conduct additional data collection. This extra 
work requires the dedication of resources, time, and personnel. The North team members 
spent a great deal of time accessing data from a variety of sources — the district 
information system, their own databases, and reading program software. They learned 
how to merge these data into a comprehensive reading database for the school. The 
survey group learned how to create new data resources by designing and conducting their 
own survey of teacher reading strategies. In the process of conducting their data inquiry, 
each team member became more familiar with the district’s information system, state 
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assessment results, and district-level resources, as well as with his or her own school- 
level databases and reading information resources. The team structure of the work groups 
facilitated the development of new knowledge of data sources within and outside the 
school building. Team members were able to maximize their time by distributing the data 
tasks and maximize their learning by sharing their results with each other. 

Fourth, there are significant problems associated with data management that 
require organization, additional resources and staffing to resolve. There is a big gap 
between data collection and data application. In between, data must be cleaned, secured, 
updated, imported into analytical software, analyzed, and formatted for reporting. To 
prevent this flow of data from becoming interrupted, the processes must be well defined, 
organized and maintained on a continual basis. Getting organized to better manage, 
maintain, and apply data will be necessary if a school is to incorporate data seamlessly 
into its professional learning and decision-making operations. The North team struggled 
with this challenge, since they were used to relying on their principal and office staff to 
procure, manage, and analyze data. The team began slowly to access data from existing 
databases, build new ones, and tackle the task of analyzing and developing reports. The 
team set up its databases with an eye toward continuous improvement and collective 
inquiry. Members of each work group learned how to set up the databases and 
spreadsheets for their data inquiry. They distributed the responsibility of learning how to 
collect, manage, and maintain the data. They designed the databases to inform current 
inquiry as well as to support future reflection and evaluation and to provide answers to 
new questions. The principal purposefully took a hands-off stance to encourage the team 
members to learn these skills on their own. At the same time, he provided technical 
assistance when needed and encouraged the team to make its own decisions regarding the 
analysis and sharing of data with other staff members. 

Fifth, staff members require additional training to improve essential analytical 
skills and abilities. Staff members need to learn how to ask better questions, how to select 
the appropriate indicators to identify strengths and weaknesses, and how to report, 
interpret, and use the results. Staff participants in the study at North stressed the 
importance of learning how to develop their analytical capacity. From the beginning, they 
took a professional interest in learning how to frame questions, read and interpret test 
results, and analyze survey data. At the time of this writing, the North team is still 
working through the analysis stage of its data inquiry process. The two work groups have 
worked with our researchers to learn how to frame their analysis, link their questions and 
goals to their data sets, and produce appropriate visual representations and reports. In the 
upcoming month, they will develop reports and recommendations based on their findings 
to share with the rest of the school staff to inspire discussions and decisions regarding 
changes in their reading program. It is too soon to tell how useful the data results and 
reports will be to North’s planning and decision-making processes. 

The sixth and final challenge for schools is to learn how to appropriately apply 
data results and make purposeful and ethical uses of information for improving teaching 
and learning. If schools have followed a continuous improvement process for planning 
and decision making, the results will be easily linked back to specific questions, goals. 
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and problems. By focusing the data analysis to target specific issues, schools will be 
poised at the end of the analytical process to make sense of and draw meaning from their 
results. The final step is to share the new information and results with staff members to 
inform school planning and decision making. The results can be used in a variety of 
ways — to identify progress, explore problems, and target strategies for change, to 
mention a few. The North team has participated in a data-inquiry process that will poise 
them for making such a strategic application of their results. Their teamwork has been 
organized in ways to address each of the school challenges for using data effectively. 
They have made significant progress in building a school-wide data culture and in 
transforming the school into a learning organization. Critical to that process was the 
development of their team as a professional learning community. In addressing the 
challenges of learning to use data effectively, the team employed all the main features of 
an effective professional learning community: 

1. Shared mission, vision and values 

2. Collective inquiry 

3. Collaborative teams 

4. Action orientation and experimentation 

5. Continuous improvement 

6. Results orientation (DuFour & Baker, 1998, pp. 25-29) 

Meeting Teacher Data Needs 

In our interviews, teachers explained that in order to learn from data and use it 
effectively, four general areas of need must be met: 1) Data inquiry needs to be a 
collaborative, shared process; 2) data needs to have a leaming/instructional focus; 3) 
professional development needs to focus on assessment literacy, alignment, decision- 
making processes, data skills, and strategy identification; and, 4) school-level 
organizational mechanisms must be in place to provide incentives and support for data 
inquiry, instructional improvement, professional development, and continuous 
improvement. 

The West Elementary School Example 

The team at West Elementary School is our most teacher-centered team, 
comprised of one teacher for every grade level in the school, and the school’s literacy 
coach and curriculum implementer (both of whom used to be teachers at the school). 
While the principal has attended many of the meetings and been very supportive of the 
study, she has confidently allocated the leadership and work for the study to the team. 
This team provides an excellent lens for investigating the role of professional learning 
communities in meeting teacher needs for learning from data and using data effectively. 

West Elementary School uses a global education curriculum that features a strong 
emphasis on basic skills. West utilizes district test scores, results from curriculum unit 
tests, assessment programs, and student portfolios. The school collects MPS Educational 
Plan data once a year; other student learning data (such as classroom-based assessments 




21 22 



Learning from Data: The Role of Professional Learning Communities 



Sarah A. Mason 



and writing samples) are collected two or three times a year. Overall, technology plays a 
limited role in data use at the school, except for some computer-generated reports. 
School-wide data are collected and managed by one or two staff members and the 
principal. MPS proficiency data is kept in paper binders. Most data analysis is limited to 
the comparison of averages and is used to guide change in the school and classrooms 
twice per year. This year, the staff conducted for the first time an objective performance 
index analysis of their TerraNova state test results. This year too, staff members report 
having access to improved data-related training, which has helped to increase 
“ownership” of data and responsibility for student learning among staff. Sharing of data 
and conclusions occurs occasionally (generally through mixed grade-level team 
discussion on district professional development days). However, staff members and the 
principal report that the process of using data to influence curriculum and instruction and 
disseminating data to relevant staff is limited. In August, 2002, West was identified for 
the first time as one of 27 Milwaukee elementary “schools in need of improvement.” 
Learning of this designation based on the 4*'^ grade students’ poor performance on the 
state assessment in mathematics, the staff reviewed the tests results and became 
convinced that the problem resided in gaps in their curriculum and/or instruction. 
Participation in our study provided an opportunity to conduct an in-depth evaluation of 
their mathematics program. 

West teachers wanted to learn about the scope and sequence of their mathematics 
curriculum from kindergarten through grade 5. They instinctively believed that many of 
the state standards were not being covered and that students may not be prepared early to 
learn skills and content presented (and assessed) in the later grades. They were also 
concerned about the pacing to the curriculum in grades 3 and 4, as students seemed 
unprepared for the state testing that occurs in November. West teachers formed a team, 
with one teacher from each grade level, to further research their student performance 
strengths and weaknesses, to align their curriculum with the state standards and state 
assessments, and to develop standards-based learning targets or performance measures 
that could be used throughout the year to measure student progress. 

One of the unmet needs teachers identified in interviews is that data inquiry needs 
to be a collaborative, shared process. Teachers said they needed to leave the isolation of 
their classrooms and have opportunities to work and learn with their colleagues. 
Developing the skills and abilities to use and apply data in a collaborative setting where 
experience can inform sharing and learning is what teachers sought. To accomplish their 
data inquiry tasks, the team at West decided to work in a collaborative manner, with a 
commitment on the part of each team member to learn new skills and processes. The 
team meets regularly and exhibits a high degree of professional involvement. Each 
member of the team contributes to discussions, sharing experiences in the classroom and 
with students. This “real world” lens helps to interpret data, validate findings, and 
legitimize decisions. The team also has fun — meetings include snacks and humor — 
making the time spent less a burden. As a result of their meetings, a sense of collegiality 
and a positive “data culture” is beginning to emerge. 
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Teachers interviewed in our studies also identified the types of data that would be 
most meaningful, and most useful, in informing improvement in the classroom. They 
said, above all, that data needed to have an instructional focus to provide feedback 
beyond the typical once-per-year assessment data and student discipline data available to 
teachers. In addition, professional development is needed and should focus on assessment 
literacy, alignment, decision-making processes, data skills, and strategy identification. 
West teachers began their data inquiry by creating their own analysis of their curriculum. 
In the process of aligning their curriculum to the state standards and assessments, they 
learned that instructional data comes in many forms and from a variety of sources. They 
learned to break down, align, and interpret content standards and curriculum objectives. 
They increased their assessment literacy by learning more about the details of how 
student learning is measured in standardized, norm-referenced tests. They even developed 
their own performance indicators to measure student progress towards the standards, 
using their classroom curriculum and assessments. The team learned these data skills 
during the course of our study; the process was intensive and required input from all 
teachers. They also expanded the scope of their data to provide information on 
curriculum and the standards and to draw connections between classroom curriculum and 
assessments to the state assessments, by which school performance is determined. 

The fourth area of need teachers identified in our interviews related to incentives 
and supports. School-level organizational mechanisms must be in place to provide 
incentives and support for data inquiry, instructional improvement, professional 
development, and continuous improvement. We have already noted that the team at West 
has made great progress in learning data inquiry processes and skills, focusing on 
improving instruction. West has also taken care to consider how these processes and 
skills can be disseminated throughout the school. Team members decided that their use of 
data and analysis needs to be shared with the entire staff and should be ongoing to 
become part of the school improvement processes on a regular basis. Both incentives and 
supports play a role in perpetuating the team’s interest in learning from data and using 
data to support school-wide change and improvement. While their initial incentive to 
embark on a data inquiry process was inspired by their designation as a school needing 
improvement, these “external pressures” were not all that sustained their inquiry. Rather, 
the team received administrative support in terms of time, access to data, authority to 
make team-based decisions, and endorsement of its work. This active support and 
recognition of hard work and progress on the part of administration became a type of 
incentive for the team to learn and grow. The team structure facilitated professional 
development of individual teachers, increasing their motivation to learn about and use 
data. The team structure also allowed teachers to think about improvement beyond the 
individual professional sphere to include organizational and school improvement. By 
expanding the sphere of influence, their data inquiry took on more meaning and became 
more important to the participants. 

The team at West operated successfully by meeting all the teacher-identified 
needs for using data and by operating as a professional learning community. The 
common focus was to evaluate the mathematics program — a process bolstered by shared 
beliefs about what needed improvement in the curriculum and in instruction. They 
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worked collaboratively, with a representative. team of committed teachers, who shared 
their experiences and learned from each other. They were open to exploration and to 
learning new skills. They have begun to apply what they learned and to share it with 
others. Team members were committed to continuous improvement, not just in individual 
classrooms but school-wide. All these traits are essential to what DuFour and Baker 
(1998) recommend as best practices for enhancing student achievement through the use 
of professional learning communities. 

Summary: Learning from Data in a Systemic Reform Context 

It is clear that using data effectively at the school level for organizational learning 
and systemic reform is a complex undertaking. Our studies of the Milwaukee district and 
schools were designed to reveal the capacity builders, barriers, needs, and conditions 
necessary for such successful data-use. From the systemic reform literature, we borrowed 
the analytical framework of human, social, organizational, and structural capital for 
evaluating the analytical capacity of districts and school. We defined success as effective 
use of data — data that is transformed into useful knowledge for affecting active reform 
and improvement of a systemic nature. From our embedded research studies, we 
identified the key challenges facing schools as they implement or improve their use of 
data. We also discovered what teachers need in order to be motivated to make sense of 
data, learn from data, and apply data in meaningful ways. Our initial intention was not to 
study professional learning communities per se, but these organizational structures 
proved time and again to match the challenges and needs of school level staff seeking to 
use and learn from data. 

In the course of our research, we have learned that in addition to building school- 
level culture, processes, and skills, schools also need to create organizational and 
structural mechanisms for using data to improve teaching and learning. We learned that 
decision making and reform capacity needed to be distributed beyond the scope of 
administrators to include classroom practitioners. We have learned that many schools are 
simply not organized to use data to improve teaching and learning. In our most recent 
study, school teams worked collaboratively to learn about continuous improvement, 
decision-making and data inquiry processes, and the analysis, application, and use of 
data. Ongoing field research has revealed that these school teams and the processes they 
employed exemplify the key characteristics of learning communities. Moreover, these 
professional learning communities appear to provide an ideal organizational structure to 
address the challenges of schools and the needs of teachers as they seek to learn from 
data and use it effectively to improve student learning (Clune & Webb, 2002; Webb, 
2001 , 2002 ). 

Since our research emphasis was on data use and not on professional learning 
communities, we did not research variations in school learning organizations and 
professional learning communities. We do not know whether a successful professional 
learning community must encompass an entire school faculty or whether a small study 
group that shares information with other staff for decision making and knowledge 
building is sufficient for building data capacity. We did not learn what it takes to 
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institutionalize and sustain effective data use in professional learning communities. 
However, now that professional learning communities have come to our attention, and 
appear to hold such promise, we intend to continue to investigate their role in facilitating 
systemic reform, organizational learning, and effective data use. 
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